bundle camx & iris-vpu & audioreach DMKS modules - #80
Closed
Christopher Obbard (obbardc) wants to merge 111 commits into
Closed
bundle camx & iris-vpu & audioreach DMKS modules#80Christopher Obbard (obbardc) wants to merge 111 commits into
Christopher Obbard (obbardc) wants to merge 111 commits into
Conversation
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
…merges in a dedicated step Previously, only one qcom-next PR could be specified per input. With this update: Multiple PR support - Accept space-separated PR numbers in the qcom-next-pr input (e.g., 12 45 78). - Iterate through each PR, fetch its head, and merge sequentially. - Abort and fail the job immediately if any merge results in a conflict. Empty input handling If no PRs are provided (empty or default value), the step is skipped gracefully without error. Separate workflow step Moves qcom-next PR merge logic out of the sync kernel step into its own dedicated step for better clarity and maintainability. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Implement dynamic logic to construct build_info file containing key environment variables and kernel build configuration parameters: JOB_ID JOB_ATTEMPT ORG_NAME REPO_NAME QCOM-BUILD-UTILS BRANCH/TAG QCOM-BUILD-UTILS HEAD SHA KERNEL BRANCH/TAG KERNEL HEAD SHA PRs FROM QCOM-NEXT PRs FROM KERNEL TOPICS To be uploaded alongside kernel deb package in the S3 bucket. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Added a temporary cron schedule for testing purposes. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
- Add a README section describing the build-kernel CI workflow, including: - Manual (workflow_dispatch) vs nightly scheduled execution at 05:00 UTC / 21:00 PST - Supported inputs, their defaults, and how they influence the build - High-level kernel build and Debian packaging pipeline using qcom-build-utils (kernel sync, optional PR integration, config enablement, kernel build, deb packaging) - Provenance tracking via build_info metadata for each run - Introduce a mermaid diagram illustrating the workflow from triggers through inputs to final outputs, to provide an at-a-glance system view - Clarify final S3 artifact layout, including bucket name and run-specific path convention (ORG_NAME/pkg/temp/REPO_NAME/RUN_ID-RUN_ATTEMPT), to improve discoverability and traceability of produced kernel .deb packages Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Added note about planned migration to native Debian tooling for kernel build orchestration and package generation. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Add a DTB manifest file defining the ordered set of device-tree blobs to be included in the combined DTB image for Qualcomm platforms. The manifest currently covers: - QCM6490 IDP - QCS6490 RB3 Gen2 - Lemans EVK - QCS9100 Ride (R0/R3) - QCS8300 Ride - Monaco EVK - QCS615 Ride - SA8775P Ride (R0/R3) - Hamoa IoT EVK This file is consumed as a configuration parameter by the combined DTB image tooling (e.g. build-dtb-image.sh in the image build pipeline) and provides a single source of truth for the DTB set and ordering used when generating dtb.bin for UEFI-based boot flows and CI builds. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Extend the pkg-linux-kernel build-kernel CI workflow to generate and publish a combined DTB image (dtb.bin) alongside the kernel Debian artifacts.
Changes:
- After the kernel build completes, invoke qcom-build-utils/kernel/scripts/build-dtb-image.sh inside the kmake-image:ubuntu-noble-arm64 container to produce dtb.bin:
-dtb-src = qcom-build utils/kernel/out/arch/arm64/boot/dts/qcom
-manifest = pkg-linux-kernel/config/dtb-manifest
-size = 4
-out = dtb.bin
- Add an in-container size sanity check that:
- Iterates all DTB entries from the manifest (skipping blank/comment lines) under the DTB source tree.
- Sums their byte sizes and compares the total against the size of dtb.bin.
- Fails the job if the sizes do not match, catching manifest/DTB mismatches early in CI.
- Refactor artifact assembly into a dedicated host-side step for
readability:
- Create qcom-build-utils/kernel/deb_artifact/.
- Copy all generated kernel .deb packages into deb_artifact/.
- Copy dtb.bin into deb_artifact/ when present to publish the combined DTB image as a first-class CI artifact.
- Keep the existing build_info generation and S3 upload logic unchanged, so the final artifact set uploaded under: ${ORG_NAME}/pkg/temp/${REPO_NAME}/${JOB_ID} ${JOB_ATTEMPT}/ now includes kernel .debs, dtb.bin, and build_info in a single deb_artifact/ directory.
This makes the combined DTB image part of the standard kernel build pipeline and ensures that the dtb.bin produced in CI is both manifest-driven and size-validated.
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Update the build-kernel workflow to source DTBs from the in-tree kernel build output instead of the out/ scratch directory.
Specifically:
- Change DTB_SRC from:
$BUILD_TOP/out/arch/arm64/boot/dts/qcom
to:
$BUILD_TOP/qcom-next/arch/arm64/boot/dts/qcom
This aligns the combined DTB image generation with the actual location of the built DTBs produced by the qcom-next kernel build.
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Temporarily disable the combined DTB image (dtb.bin) build step for `schedule`-triggered runs in the build-kernel workflow by guarding the step with `if: ${{ github.event_name != 'schedule' }}`.
Rationale:
- The dtb.bin generation relies on build-dtb-image.sh and related DTB tooling that currently exists only on a feature branch of qcom-build-utils.
- Scheduled jobs consume the default `main` for qcom-build-utils, which does not yet contain the DTB image tooling, causing the scheduled pipeline to fail when attempting to build dtb.bin.
With this change:
- `workflow_dispatch` runs (where we can explicitly select a branch of qcom-build-utils that includes the DTB tooling) still build and publish dtb.bin alongside kernel .deb artifacts.
- Nightly `schedule` runs continue to exercise the kernel build and packaging paths without being blocked by the in-flight combined-DTB tooling work.
This guard is intended as a temporary measure until the DTB image build logic is merged into the qcom-build-utils mainline and becomes available to scheduled CI runs.
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
…ed to qcom-distro-images workflow Revert the last six commits to remove the combined DTB image build from the kernel repository. DTB image generation is now handled by the qcom-distro-images workflow. Reverted range: HEAD~6..HEAD Reverted commits: - 85851ca ci: gate combined DTB image build on non-scheduled runs - dfa70e1 Update dtb-manifest to temporarily include single dtb only - 3d86510 ci: fix DTB source path for combined DTB image build - 5c48c7a Remove combined dtb size sanity check - da87d29 ci: integrate combined DTB image build into kernel workflow - b1a366f config: add dtb-manifest for combined DTB image generation
Extends the `build-kernel` workflow to accept an optional user-defined `kernel-url`. This enhancement allows the pipeline to ingest and build kernel sources from other repositories outside of the standard `qualcomm-linux` GitHub hierarchy. Key changes: - Introduced `kernel-url` input to the workflow dispatch trigger. - Implemented mutually exclusive sync logic: - **Default:** Retains existing behavior (syncs from `qualcomm-linux/kernel`, applies PR merges/patches) when no URL is provided. - **Custom:** Clones directly from the provided `kernel-url` and explicitly skips PR merge and patch application steps to ensure source integrity. - Refactored the local workspace directory structure, renaming `qcom-next` to `qcom-kernel` to decouple the build environment from specific branch naming conventions. - Updated all downstream path references and config scripts to align with the new workspace layout. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Updates the `Generate build_info metadata` step to include the `CUSTOM KERNEL URL` field. This ensures that the source origin of the build artifacts is explicitly traceable for auditing and debugging purposes, particularly when non-default repositories are used. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Adds a workflow_dispatch-only workflow for testing the debian kernel packaging work on feat/kernel-debian-source-packaging before it merges. To use: Actions → build-kernel-deb → Run workflow → select branch feat/kernel-debian-source-packaging. Signed-off-by: Adam Bickett <abickett@qti.qualcomm.com>
Signed-off-by: Adam Bickett <abickett@qti.qualcomm.com>
- Add input box for pkg-linux-qcom PR and workflow step to handle application of self-PRs - Add workflow step to extract REPO and ORG name - Switch to using docker-pkg-build main repo/branch - Add workflow step to upload to s3 bucket location Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
…nner Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
…ch or commit Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
… steps
Restructure the build-kernel-deb workflow so that kernel source setup is
handled by explicit workflow steps rather than being delegated to
build-kernel.sh. The script is now invoked with --local-source, receiving
a fully prepared kernel tree from the workflow.
New steps added:
Sync kernel source
Resolves the target ref (latest qcom-next-* tag via git ls-remote, or
the kernel-branch input as-is), performs a shallow clone into
$GITHUB_WORKSPACE/kernel-source, and exports KERNEL_DIR, KERNEL_REF,
and KERNEL_SHA to the job environment for use by downstream steps.
Merge qcom-next PR patches (conditional: qcom-next-pr != '')
For each space-separated PR number, fetches pull/<N>/head from the
kernel remote and merges it with --no-commit. Merge conflicts abort
the build immediately. Already-merged or fast-forwarded PRs emit a
warning and are skipped cleanly.
Apply kernel-topics PR patches (conditional: kernel-topics-pr != '')
For each space-separated PR number, downloads the GitHub-generated
.patch file from qualcomm-linux/kernel-topics and applies it with
git am, preserving authorship and commit messages. Patch failures
abort the build immediately.
The Build kernel package step is updated to pass --local-source instead
of --repo/--branch/--tag/--latest-tag. build-kernel.sh skips all git
operations and proceeds directly to debian/ injection, prepare, and build.
build-kernel.sh is not modified. Its clone/fetch capability is preserved
for developer use; CI simply uses the --local-source entry point.
Two new workflow_dispatch inputs are added:
qcom-next-pr space-separated PR numbers from qualcomm-linux/kernel
kernel-topics-pr space-separated PR numbers from qualcomm-linux/kernel-topics
Both default to empty and are no-ops when not supplied, preserving full
backward compatibility with existing runs.
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Christopher Obbard (obbardc)
force-pushed
the
wip/obbardc/more-dkms
branch
from
September 1, 2026 08:48
8740d78 to
b607795
Compare
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 08:55 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 08:57 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 08:57 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 08:57 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 14:53 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 14:53 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 14:53 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 14:53 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 15:16 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 15:17 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 15:17 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 1, 2026 15:18 — with
GitHub Actions
Failure
Contributor
Author
|
Seems like PR builds don't include "qli-staging" repo in the deps. Will resolve in separate PR. |
The packaging (debian/, prepare-source.sh, build-kernel.sh) and the CI
generator (.github/workflows/, ci/) have lived on two branches whose
histories only meet at the initial commit, so every build had to check
the two out separately and keep their assumptions in sync by hand.
This split has worked out in practice to not be ideal: changes which
touch both the packaging and the CI generator will need two separate PRs
which have to be kept in sync. Instead of this extra complication, merge
the packaging and CI branches so that PRs may target just one branch
and have the ability to modify both packaging and CI.
Merge the packaging branch into the CI branch to keep the 107 commits
of packaging history.
Two files conflict during the merge:
- README.md: The two branches hold different documents rather than
two versions of the same file; so the packaging README.md is moved to
debian/README.md and the top-level README keeps the generator focus.
Links between the two files follows in a separate commit; any
improvements/deduplication will follow in separate PRs.
- .github/workflows/qcom-preflight-checks.yml. Deleted from the main
branch in 18b0905 and modified since on the packaging branch; the
deletion stands for now.
CONTRIBUTING.md and SECURITY.md merge to the versions on the main
branch: the packaging branch never touched them after the fork point,
so the <REPLACE-ME> placeholders 9af567c filled in stay filled in.
pr-build.yml comes across as the packaging branch left it, still pinned
to main and to a separate ci-ref. Rewiring it (and the rest of the
cross-branch plumbing) follows in a separate commit.
Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
The merge left two READMEs describing two halves of one repository with no way to get from either to the other and debian/README.md still opening as though it were a repository of its own. Point each at the other and reword them slightly to describe their intention. No content moves between them yet as they are different documents. Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
With debian/ and ci/ now on one branch, the cross-branch plumbing that kept them together has nothing left to do. Every build leg took its packaging from qcom/debian/latest and its CI scripts from a separate ci-ref, which meant a matrix change and the packaging change it depended on could land in either order and only meet at build time. Drop pkg_linux_qcom_ref from the matrix and both the pkg-linux-qcom-ref and ci-ref inputs from build-kernel-deb.yml and build-kernel-ubuntu.yml, and collapse the prepare job's two checkouts into one. That checkout now carries no ref at all, leaving actions/checkout on github.sha: the commit the run was dispatched from, or the one the calling workflow runs at. No ref override is left, so a build is always the packaging and the CI of a single commit. Drop self-pr with it. It applied a pkg-linux-qcom PR on top of the packaging checkout, which is what pr-build.yml now does for every PR by construction. Pointing a dispatched build at a PR is worth having back as a first-class input, but not as a knob that fetches over the ref the rest of the run is pinned to; leave it for a follow-up. The CI scripts are 100755 in the tree, so dropping the copy step's chmod loses nothing. pr-build.yml arrived with the merge as the packaging branch left it and is adapted to the same model: it triggers on PRs to main, calls ./.github/workflows/build-kernel-deb.yml rather than pinning the reusable workflow to @main, and resolves the matrix from the PR head instead of main. A PR is now tested entirely against its own commit. Checking main out for the matrix no longer isolated anything, since the build legs run the PR's ci/ scripts either way. Daily and Release revisions are unchanged; resolve-matrix.sh emits the same debian_revision for all nine legs. Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
…-main Merge packaging branch into the CI branch
…tely
BUILD_EXCLUSIVE_CONFIG may name several kernel options, space separated, all
of which must be set for dkms to build the module. The gate report tested the
whole string as a single symbol, which never matches, so every gate was
reported as unset:
| CONFIG_ARCH_QCOM CONFIG_PM_DEVFREQ CONFIG_SYNC_FILE is NOT set
That is worse than saying nothing: it accuses the kernel config whatever the
real cause was, and it did exactly that while the actual failure was a broken
source tree. Both kgsl and iris-vpu declare multi-option gates, so the report
was wrong for every module that reached it.
Evaluate each option separately.
Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
Add camx to debian/dkms-modules so the Qualcomm camx driver is built against the kernel being packaged and shipped inside linux-image-<KVER>-qcom and declare camx-dkms in Build-Depends. Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
Add iris-vpu to debian/dkms-modules so the Qualcomm video accelerator driver is built against the kernel being packaged and shipped inside linux-image-<KVER>-qcom and declare iris-vpu-dkms in Build-Depends. Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
Add audioreach to debian/dkms-modules so the audioreach driver is built against the kernel being packaged and shipped inside linux-image-<KVER>-qcom and declare audioreach-dkms in Build-Depends. Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
Christopher Obbard (obbardc)
force-pushed
the
wip/obbardc/more-dkms
branch
from
September 2, 2026 19:52
75ff934 to
de468f9
Compare
Comment on lines
+204
to
+208
| - name: Checkout pkg-linux-qcom | ||
| # debian/ and ci/ live on the same branch, so one checkout supplies | ||
| # both the packaging and resolve-kernel-ref.sh, derive-localversion.sh, | ||
| # derive-debian-revision.sh and the suite_suffix_mapping. | ||
| uses: actions/checkout@v4 |
Comment on lines
+210
to
+215
| - name: Checkout docker-pkg-build | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: qualcomm-linux/docker-pkg-build | ||
| ref: main | ||
| path: docker-pkg-build |
Comment on lines
+80
to
+88
| - name: Checkout debusine-action helpers | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| repository: qualcomm-linux/debusine-action | ||
| ref: ${{ env.DEBUSINE_ACTION_REF }} | ||
| path: debusine-action | ||
| fetch-depth: 1 | ||
| sparse-checkout: | | ||
| lib |
Comment on lines
+150
to
+158
| - name: Checkout debusine-action helpers | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| repository: qualcomm-linux/debusine-action | ||
| ref: ${{ env.DEBUSINE_ACTION_REF }} | ||
| path: debusine-action | ||
| fetch-depth: 1 | ||
| sparse-checkout: | | ||
| lib |
Comment on lines
+42
to
+46
| - name: Checkout pkg-linux-qcom | ||
| uses: actions/checkout@v4 | ||
|
|
||
| # Pinned to @main while native kernel-build support is still landing | ||
| # upstream in qualcomm-linux/docker-pkg-build. |
Comment on lines
+47
to
+52
| - name: Checkout docker-pkg-build | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: qualcomm-linux/docker-pkg-build | ||
| ref: main | ||
| path: docker-pkg-build |
| # Steps represent a sequence of tasks that will be executed as part of the job | ||
| steps: | ||
| # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it | ||
| - uses: actions/checkout@v4 |
Comment on lines
+59
to
+66
| - name: Checkout qcom-build-utils | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: qualcomm-linux/qcom-build-utils | ||
| ref: ${{ inputs.qcom-build-utils-ref }} | ||
| token: ${{ secrets.DEB_PKG_BOT_CI_TOKEN }} | ||
| path: ./qcom-build-utils | ||
| fetch-depth: 1 |
| outputs: | ||
| matrix: ${{ steps.set-matrix.outputs.matrix }} | ||
| steps: | ||
| - uses: actions/checkout@v4 |
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 2, 2026 20:03 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 2, 2026 20:04 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 2, 2026 20:04 — with
GitHub Actions
Failure
Christopher Obbard (obbardc)
had a problem deploying
to
Staging
September 2, 2026 20:05 — with
GitHub Actions
Failure
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tested with local kernel build with additional DKMS modules.
Will NOT build on resolute, due to camx-dkms (and others) not available.
Will NOT build on debian, due to audioreach-dkms not available in all branches.